Removing Weather Effects from Monochrome Images

Images of outdoor scenes captured in bad weather suffer from poor contrast. Under bad weather conditions, the light reaching a camera is severely scattered by the atmosphere. The resulting decay in contrast varies across the scene and is exponential in the depths of scene points. Therefore, traditional space invariant image processing techniques are not sufficient to remove weather effects from images. In this paper, we present a fast physics-based method to compute scene structure and hence restore contrast of the scene from two or more images taken in bad weather. In contrast to previous techniques, our method does not require any a priori weather-specific or scene information, and is effective under a wide range of weather conditions including haze, mist, fog and other aerosols. Further, our method can be applied to gray-scale, RGB color, multi-spectral and even IR images. We also extend the technique to restore contrast of scenes with moving objects, captured using a video camera. 1 Towards Weather-Free Vision Most outdoor vision applications such as surveillance, target tracking and autonomous navigation require robust detection of image features. Under bad weather conditions, however, the contrast and color of images are drastically degraded. Hence, it is imperative to remove weather effects from images in order to make vision systems more reliable. Unfortunately, the effects of bad weather increase exponentially with the distances of scene points from the sensor. As a result, conventional space invariant filtering techniques fail to adequately remove weather effects from images. Recently, there has been an increased interest in the image processing and vision communities on issues related to imaging under bad weather. Kopeika et al [5, 17] de-blur atmospherically degraded images using a weather-predicted atmospheric modulation transfer function, and an a-priori estimate of the distance from which the scene is imaged. Oakley et al [12, 15] describe a physics based method to restore scene contrast without using predicted weather information. However, they assume that scene depths are known beforehand, and approximate the distribution of radiances in the scene by a single gaussian with known variance. Narasimhan and Nayar [10] analyze the color variations in the scene under different weather conditions based on the dichromatic atmospheric scattering model proposed in ∗This work was supported in parts by a DARPA Human ID Contract (N00014-00-1-0916) and an NSF Award (IIS-99-87979). [11]. Using constraints on scene color changes, they compute complete 3D structure and recover clear day scene colors from two or more bad weather images. However, they assume that the atmospheric scattering properties do not change with the wavelength of light. This property holds over the visible spectrum only for certain weather conditions such as fog and dense haze. Furthermore, the dichromatic model is ambiguous for scene points whose colors match the color of fog or haze. Polarizing filters have been used widely by photographers to reduce haziness in images. However, polarization filtering alone does not ensure complete removal of haze. Schechner et al [13] further analyzed 2 or more polarization filtered images to compute scene structure and dehaze images. Another work by Grewe and Brooks [2] uses wavelet based fusion of multiple bad weather images to get a less blurred image. In this paper, we present a physics based method to restore contrast completely from two or more images taken in bad weather. A monochrome atmospheric scattering model that describes how scene intensities are effected by weather conditions is presented. This model is valid in both the visible and near-IR spectra, and for a wide range of weather conditions such as mist, haze, fog and other aerosols. Based on this model, an automatic algorithm to recover complete scene structure from two images taken under different weather conditions is presented. Using the computed structure, contrast is restored from a single image of the scene. We extend our algorithms to handle video and describe a simple heuristic to restore contrasts of moving objects in the scene whose depths are unknown. The entire analysis in this paper is done for monochrome (narrow spectral band) images. However, we show that our methods can be applied to images taken using gray-scale, wide-band RGB, multi-spectral and also narrow-band IR cameras. The effectiveness of these sensors under various weather conditions is discussed. 2 Atmospheric Scattering Models Scattering of light by physical media has been one of the main topics of research in the atmospheric optics and astronomy communities. In general, the exact nature of scattering is highly complex and depends on the types, orientations, sizes and distributions of particles constituting the media, as well as wavelengths, polarization states and directions of the incident light [1, 3]. Here, we focus on two models attenuation and airlight, that form the basis of our work. 2.1 Attenuation and Airlight The attenuation model describes the way light gets attenuated as it traverses from a scene point to the observer. The attenuated irradiance is given by (see [7, 10]), Edt(d, λ) = E∞(λ) r(λ) e−β(λ)d d2 . (1) where, d is the depth of the scene point from the observer and λ is the wavelength. β(λ) is called the scattering coefficient of the atmosphere; it represents the ability of a unit volume of atmosphere to scatter light in all directions. β(λ)d is called the optical depth of the scene point. E∞ is the horizon brightness and r is a function that describes the reflectance properties and the sky aperture1 of the scene point. The second atmospheric scattering model we consider is called the airlight model. The airlight model quantifies how a column of atmosphere acts as a light source by reflecting environmental illumination towards an observer. The irradiance due to airlight is given by (see [6]), Ea(d, λ) = E∞(λ) (1 − e−β(λ)d) . (2) The total irradiance E received by the sensor is the sum of irradiances due to attenuation and airlight respectively : E(d, λ) = Edt(d, λ) + Ea(d, λ) . (3) 2.2 Wavelength Dependence of Scattering Generally different wavelengths of light are scattered differently by atmospheric particles. Interesting atmospheric phenomena such as the blueness of the sky and the bluish haze of distant mountains are examples of the wavelength selective behavior of atmospheric scattering [4, 8]. In these cases, the blue wavelengths are scattered more compared to other visible wavelengths. On the other hand, fog and dense haze scatter all visible wavelengths more or less the same way. Over the visible spectrum, Rayleigh’s law of atmospheric scattering provides the relationship between the scattering coefficient β and the wavelength λ [6] :